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DECISION  THEORY  RESEARCH 


1.0  INTRODUCTION 


The  purpose  of  this  report  is  to  provide  abstracts  with  ^ 

suitable  references  reflecting  significant  accomplishments 
in  each  of  nine  decision  theory  research  areas  during  the 
period  October  1,  1972,  through  June  30,  1975.  To  this  end, 
frequent  references  are  made  to  four  technical  progress 
reports  published  by  Decisions  and  Designs,  Incorporated 
(DDI)  during  the  foregoing  period  which  more  fully  report 
the  results  of  this  research.  For  convenience  and  brevity, 
references  throughout  this  report  to  any  one  of  these  four 
technical  reports  will  be  parenthetical,  indicating  volume 
number  in  Roman  numerals,  and  section  and  sub-section  numbers 
in  Arabic  numerals;  e.2.w  (1/  3.2)  refers  to  Technical  ^ 

Report  Number  I,  Section  3,  Sub-Section  2.  A full  description 
of  these  technical  reports  is  provided  in  the  Bibliography. 

Additional  references  are  also  identified  in  the  text  and 
fully  described  in  the  Bibliography. 

The  research  reflected  in  this  report  had  four  primary 
tasks;  namely,  to 

Task  1:  Investigate  procedures  for  improving  human 

judgments  of  probabilities  and  utilities  for  decision- 
making ; 

Task  2;  Conduct  problem-oriented  workshops  for  Depart- 

of  Defense  (DoD)  personnel  in  which  the  potential  value 
of  decision  analysis  techniques  is  displayed  to  decision- 
makers by  showing  them  how  these  techniques  can  be  applied 
to  real  problems; 

Task  3:  Prepare  a handbook  on  decision  analysis  designed 

for  the  malnager  or  staff  responsible  for  organizing  and 
managing  a decision  analysis,  rather  than  for  the  decision- 
analytic  technician;  and 

Task  4;  Conduct  research  on  decision  analysis  procedures 
in  connection  with  current  intelligence  analysis  and  with 
scientific  and  technical  intelligence  analysis  in  co-opera- 
tion with  the  Defense  Intelligence  Agency  (DIA)  and  the 
Naval  Intelligence  Support  Center  (NISC) . 
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2.0  ASSESSMENT  AND  FORECASTING 


This  category  of  research  includes  the  problem  of 
assessing  and  expressing  uncertainty,  training  and  evaluating 
analysts  in  assessing  probabilities,  and  developing  techniques 
for  modeling  the  current  state  of  affairs,  and  forecasting 
the  likelihood  of  future  events. 


Training  Analysts  in  Probability  t.: 
Techniques  of  Probability  Analysis 


sessments 


Two  experiments  on  probability  assessment  were  conducted 
in  1972  at  the  Defense  Intelligence  School  of  the  DIA  using 
approximately  one  hundred  student  analysts  as  subjects.  One 
experiment  was  designed  to  evaluate,  in  a static  situation, 
whether  odds  are  more  extreme  than  probabilities,  when 
individuals  assess  likelihood  and  also  whether  the  odds  mode 
is  more  accurate.  The  second  experiment  evaluated  the 
relative  merits  of  two  different  response  modes  that  could 
be  used  for  assessing  the  probability  of  events  that  lie 
along  a continuum.  Examples  of  continua  used  in  intelligence 
estimates  are  the  speed  of  an  aircraft,  the  range  of  a 
missile,  and  the  number  of  personnel  in  a division.  (I, 

1.2.3  and  I,  2.3,  and  also  References  11.2.2.12  and  11.2.2.13 
in  the  Bibliography) 

This  early  work  led  to  a program  to  train  analysts 
assigned  to  a division  of  the  DIA  in  the  use  of  probability 
assessments.  (I,  2.3)  After  they  were  trained  and  had 
become  experienceu  in  probability  assessments,  data  were 
collected  for  an  18-month  analysis  period  to  see  how  accurately 
the  analysts  were  performing.  The  correlation  between  what 
the  analysts  estimated  and  what  events  actually  occurred  was 
exceedingly  high  and  demonstrated  that  properly  trained 
analysts  can  become  highly  proficient  in  assessing  numerical 
probabilities.  (II,  5.1.2) 

Additional  work  with  probability  assessments  led  to  the 
development  of  interactive  computer  programs  to  assist  and 
train  analysts.  One  program  implements  a proper  scoring 
rule  tool  for  calibrating  analysts  who  assess  probabilities 
and  another  implements  a method  for  directly  assessing 
probabilities  of  events  that  lie  along  a continuum.  (V, 

5.2,  and  also  Reference  11.2.2.1  in  the  Bibliography) 

Intelligence  "order  of  battle"  problems  require  rules 
and  methods  for  obtaining  and  combining  distributions. 

Research  shows  that  intuitive  approaches  to  target  distribu- 
tions are  often  inaccurate.  A m.odel  reflecting  this  research 
is  being  developed  for  intelligence  analysts  to  use  in 
assessing  uncertainty  associated  with  personnel  strength 
estimates . 
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Working  with  intelligence  analysts  in  the  Central 
Intelligence  Agency  (CIA)  and  the  DIA,  DDI  personnel  conducted 
a series  of  case  studies  involving  actual  intelligence 
analysis  problems.  These  case  studies,  some  of  which  are 
described  below,  illustrate  the  use  of  personal  probabilities, 
both  to  quantify  opinions  of  uncertainty  and  to  describe 
linkages  among  data,  events,  and  indicators  and  the  hypotheses 
of  interest.  This  work  made  it  possible  for  analysts  to 
construct  quantitative  models  of  their  problems  and  to 
communicate  their  perceptions  of  the  f i jblems  to  others. 

2.2.1  The  Cyprus  analysis  - This  analysis  had  two 

principal  objectives:  to  present  an  analysis  of  the  political 

situation  on  Cyprus  and  to  display  the  analysis  to  the  user 

in  such  a way  as  to  provide  a justification  for  the  results. 
Employing  probability  diagrams  and  a Markov  model,  the 
analysis  developed  a specific  probability  and  associated 
rationale  for  the  hypothesis  that  there  v;ould  be  widespread 
hostilities  during  the  next  six  months.  (II,  2.2.2,  and 
also  Reference  11.2.1.1  in  the  Bibliography) 

2.2.2  The  Sino-Soviet  border  analysis  - This  analysis 
hypothesizes  a lumber  of  attack  scenarios  and  structures  the 
relationships  among  such  factors  as  the  build-up  of  forces, 
political  factors  (internal  and  external) , and  border  inci- 
dents, and  each  of  six  explicitly  stated  hypotheses  concerning 
conflict  between  the  two  nations.  Personal  probabilities 
reflecting  the  state  of  knowledge  of  different  specialists 
are  generated  by  using  structured  procedures  to  encode  the 
exoert  judgement  of  analysts  and  groups  of  analysts.  (Reference 

11.2.1.2  in  the  Bibliography) 

2.2.3  The  North  Vietnamese  (NVN)  analysis  - This 
analysis,  which  involved  weekly  forecasts,  charted  the 
impact  of  incoming  evidence  in  the  form  of  a numerical 
li):elihood  ratio  that  measured  the  degree  to  which  a particu- 
lar hypothesis  concerning  an  NVN  attack  was  favored.  In 
this  analysis,  intelligence  personnel  learned  to  display  the 
impact  of  new  data  on  the  prior  odds  through  the  use  of  a 
"log  odds"  chart.  Working  with  analysts  who  had  no  previous 
experience  in  Bayesian  techniques,  DDI  personnel  confirmed 
the  fact  that  the  log  odds  procedure  provided  a viable 
approach  for  training  analysts  to  assess  likelihood  ratios. 

(II,  5.1  anc  III,  5.1,  and  also  Reference  11.2.1.7  in  the 
Bibliography) 

2.2.4  Other  intelligence  analysis  projects  - In  the 
case  of  many  complex,  technical  intelligence  inference 
problems,  it  is  unlikely  that  any  one  individual  has  the 
necessary  experience  to  relate  all  of  the  detailed  data 
concerning  a nation's  research  and  developm.ent  programs  to 
upper-level  hypotheses.  To  solve  this  problem,  a number  of 
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3.0  THE  EVALUATION  OF  ALTERNATIVE  SOLUTIONS 


The  major  problem  of  evaluation  is  to  assign  a meaningful 
me  .sure  of  benefit  to  alternatives.  Procedures  are  needed 
fo  ■ the  development  and  application  of  scenarios  and  for 
going  from  broad  goals  to  the  specification  of  requirements 
and  the  selection  of  systems.  This  research  led  to  the 
de  '’elopment  of  procedures  that  can  be  used  to  translate 
technical  considerations  into  m.eaningfu'’  measures  of  ufer- 
oriented  worth. 

3 . 1 Operational  Test  and  Evaluation  (OT&E) 

Design  to  cost  procurement  policies  often  require  OT&E 
personnel  to  decide  w lether  prototype  testing  is  necessary 
to  determine  the  acce >tability  of  a proposed  system  and;  if 
SO/  how  much  testing  and  whether  or  not  subsequent  testing 
is  required  before  procurement/  due  to  modifications  which 
resulted  from  the  initial  tes'.ing.  Such  a complex  decision 
has  been  systematized  by  a decision  analytical  model  which 
incorporates  a hierarchical  evaluating  mechanism  within  the 
framework  of  a value-of-information  analytical  modal.  A 
case  study  for  the  Commander / Naval  Operational  Test  and 
Evaluation  Force  (OPTEVFOR)  which  explained  the  rationale 
for  this  decision  analytical  mode]  v,as  followed  by  a pilot 
pro  ect  evaluating  the  suitability  of  the  methodology  for  a 
specific  weapon  system  (II/  2.2.4) 

3 . 2 Allocation  of  Decision  Analysis  Resources 

A system  has  been  developed  for  a] locating  funds  and 
evaluating  proposals  not  only  for  decision  analysis  research/ 
but  also  more  generally  for  research  in  many  other  disciplines . 
The  system  divides  the  areas  of  research  top: cally / establishes 
a standard  allocation  of  resources  and  a corresponding 
benefit/  and  then  varies  the  allocations  to  determine  the 
changes  in  benefit  along  a "response  curve".  A computer  ^ 
program  ha'  been  developed  which  permits  specific  evaluations 
and  sensi t-i.vity  analyses  of  research  proposals  in  terms  of 
the  allocations  of  program  funds.  (Ill/  2.1  and  IV/  3.7/ 
and  also  Reference  11.2.2.7  in  the  Bibliography) 

3 . 3 Design  to  Cost  Contractor  Selection 

A multi-attribute  utility  model  was  developed  to  quantify 
user  preferences  and  relate  those  preferences  to  technical 
system  cliaracteristics  in  the  design  to  cost  competitive 
acquisition  of  an  Electronic  Warfare  (EW)  Suite.  The  evalua- 
tion model  was  structured  to  decompose  high-level  operational 
requirements  into  more  detailed  performance  requirements  and 
to  relate  these  to  the  degree  of  technical  performance 
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procosod  by  oacn  of  the  cortractors.  The  results  of  operating 
the  model  in  an  iterative  mode  wore  used  to  select  the  most 
promising  approaches  and  to  reduce  the  number  of  contractors 
selected  to  build  prototype  EW  systems  to  two. 
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4.0  POLICY  FORMULATION  AND  IMPLEMENTATION 

The  policy  analyst  charged  with  making  national  policy 
recommendations  is  required  to  review  a wide  range  of  uncer 
tain  factors,  examine  costs  and  benefits, 

alternative  courses  of  actions.  Research  conducted  by  DDI 
developed  structured  approaches  that  clarified 
simplified  presentation  by  using  quantified  decision  models. 
Policy-making  becomes  more  manageable  for  analyst  and  decisio 
maker  alike  since  all  assumptions  and  judgments  are  made^ 
explicit,  can  be  readily  examined,  and,  if  necessary,  re 
evaluated  and  revised. 

4.1  Evaluating  Foreign  Policy  Alternatives  for  A Midea^ 

Oil  Producer 

Negotiating  stratecies  to  ensure  a continuing  and 
expanding  supply  of  oil  from  a Mideast  oil-producing  country 
concerned  staff  members  of  the  National  Security  Council 
(NSC).  DD]  developed  a formal  structure  for  incorporating^ 
data  from  economic.  Mideast,  oil,  and  CIA  experts.^  Ana  ysis 
of  the  problem  led  to  an  initial  model  for  evaluating  three 
quite  different  negotiating  strategies.  The  first  was  a 
"base"  option  requiring  no  change  in  present  U.S.  ^oiicies^  ^ 
toward  the  Mideast.  The  second  was  an  extreme  option  requiring 
a radical  change  in  current  U.S.  policies  to  accommodate _ the 
interests  of  the  oil-producing  country  in  every  way  possible. 
The  third  was  an  intermediate  option  requiring  some  changes 
in  present  policies.  The  model  incorporated _ not  only  these 
options,  but  also  their  consequences;  four,  in  particular: 

1)  the  effect  on  balance  of  payments;  7.)  the  impact  on  U.S. 
relations  with  Europe  and  Japan;  3)  the  impact  on  U.S. 
relations  with  Israel  and  on  U.S.  pro -Israeli  sentiment;  and 
4)  the  effect  on  other  oil  producers.  This  study  indicated 
that  concern  over  the  supply  of  oil  required  a change  in 
present  U.S.  policy,  but  that  an  unfavorable  impact  on 
Allied  relations  and  pro-Israel  sentiment  made  the  less 
extreme  option  more  attractive  to  policy  makep._  A classified 
technical  report  (Reference  11. 2. 1. 6 in  the  Bibliography) 
highlights  a new  technique  for  simplifying  prediction  of 
interrelated  continuous  variables.  An  unclassified  version 
of  this  report  was  presented  at  the  Decision  Analysts 
Conference  in  1973  and  during  numerous _ brief ings , such  as 
one  f o ' the  Senior  Seminar  at  the  Foreign  Affairs  Institute, 
have  been  given.  (IV,  3.1) 

4 . 2  Export  Controls  on  Sales  of  Computers  to  the  Soviet  Blo_c 

The  level  of  embargo  that  should  be  established  for 
computers  sold  to  the  Soviet  Bloc  concerned  the  Assistant  to 
the  President  on  National  Security  Affairs.  At  his  direction. 
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5.0  VALUE-OF- INFORMATION  PROBLEMS 


Research  in  this  area  focused  primarily  on  the  problem 
of  allocating  resources  for  the  collection  of  intelligence 
information.  The  technical  approach  was  based  upon  a principle 
of  decision  theory  which  states  that  it  is  worthwhile  to  buy 
(collect)  information  only  if  it  serves  to  change  behavior, 
and  that  the  value  of  the  information  is  the  difference 
between  the  expected  values  of  the  old  and  new  behaviors. 

Some  of  the  specific  decision  analytic  techniques  developed 
and  applied  to  a variety  of  value-of-information  problems 
are  described  below.  The  ultimate  objective  is  to  improve 
the  utility  of  information  collected  and  analyzed  as  part  o 
the  national  decision-making  process.  (I,  1.2.2,  and  IV, 

4.3) 


5 . 1 Evaluating  Reconnaissance  Vehicles 

This  evaluation  was  designed  to  establish  a functional 
relationship  between  the  intelligence  value  of  a given 
'ollection  platform  and  the  number  of  collection  hours  flown 
per  month.  The  analysis  was  performed  with  DIA  analysts  and 
was  based  on  the  premise  that  information  has  value  to  the 
extent  that  it  satisfies  an  important  and  valid  intelligence 
production  requirement.  The  analysis  displayed  the  number 
of  hours  for  which  the  value-to-cost  ratio  was  greatest  and 
least,  and  for  which  the  number  of  hours  flown,  if  doubled, 
wou''d  yield  seven  times  the  previous  value  of  intelligence 
coljected.  The  study  also  included  a number  of  techinques 
for  evaluating  individual  reconnaissance  reports.  (II, 

2.2.3) 

5 . 2 Evaluating  Levels  of  Collection  Effort 

This  evaluation  resulted  in  the  development  of  a decision 
diagram  structuring  an  evacuation  problem  that  could  arise 
if  hostilities  broke  out  between  Greek  and  Turkish  Cypriots. 
The  problem  was  whether  or  not  gathering  new  information 
about  the  situation  (and,  if  so,  how  much)  would  significantly 
improve  decisions  with  regard  to  the  evacuation  of  personnel. 
As  part  of  the  analysis,  a perfect  information  model  was 
constructed,  and  the  output  (a  value  expressed  in  dollars 
for  perfect  information  about  what  will  happen)  placed  an 
upper  limit  on  expenses  for  increased  collection  of  informa- 
tion. The  results  indicated  that  the  cost  of  an  increased 
collection  effort  for  this  scenario,  even  if  it  produced  a 
perfect  report,  should  not  exceed  $360,000.  (II,  2.2.2,  and 
also  Reference  11.2.1.1  in  the  Bibliography) 
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5.3  Alloca tion  of  Intelligence  Re so urces 

The  allocation  of  intelligence  resources  is  based  on 
the  sequential  judgments  of  expert  budget  and  programming 
OMSome!  at  vakoGs  levels  within  and  among  departments  and 
aaencies  until  the  allocatioh  is  finally  approved  by  the 
ExeSSate  and  Legislative  Branches.  In  general  the  examina- 
tion  of  alternative  allocations  during  the  normal  budget 
cycle  is  made  without  a c ost-to-benef it  analysis. 

This  analysis  described  the  national  decision-making 
process  in  terms  of  the  most  important  decisions  that  senior 
officials  have  to  make  and  calculated  the  value  of  good 
intelligence  in  making  those  decisions.  The  model  consist 
of  standard  decision-analytic  procedures  modified  in  ^ few 
important  respects.  1.  example,  the  model  accounts  for 
fact  that  a decision-maker  may  act  less  on  the  bas is  of  an 
accurate  intelligence  forecast  than  on  the  basis  of  political 
or  budgetary  considerations. 

During  the  study,  a workshop  was  conducted  with  analysts 
from  DIA  who  were  experts  in  both  the  collection  an  pro  uc 
tion  areas  of  intelligence.  Their  inputs  were  important, 
since  the  model  makes  it  possible  to  examine  how  change.,  i 
the  mix  of  collection  systems  directly  affect  the  quality 
Se  ?i^ished  intelligence  product.  Results  from  the  model 
are  expressed  in  terms  of  the  expected  dollar  benefit  of  a 
good  d^ision  that  results  from  having 

intelligence.  Refinements  also  make  it  possible  to  compare 
the  value  of  individual  collection  systems  in  terms  of  thei 
respective  cost-to-benef it  ratios.  (111,2.1) 

5 . 4 Defe  ise  Attache^  System  Analysis 

This  case  study  involved  an  analysis  of  the  impact  on  ^ 
intelligeice  of  changes  in  the  budget  of  the  Defense  Attache 
Syslem.  'he  problem  was  to  assess  the  value  of  the  informs- 
tion  coll'icted  by  each  overseas  attache  office  and  to  relate 
it  to  the  known  cost  of  that  office. 

In  order  to  obtain  assessments  of  the  importance 
each  country  where  an  attache''  office  was 

value  of  different  types  of  information  obtainable  in  each 
coun?r?,  a "boot-stripping"  procedure  was  used.  Intelligence 
and  research  analysts  established  the 
each  of  several  major  areas  of  the  world  and 
importance  of  countries  within  these  areas.  Multiple 
regression  analysis  was  then  used  to  obtain  ^ 

fitted  weights  were  used  to  generate  a new  set  of  values  for 
each  country.  Once  these  values  were  obtained,  they  were 
weighted  by  the  probability  that  the  desired  information 
would  develop  within  the  country  and  the  probability  that 
the  attach^  in  the  normal  course  of  his  duties  would  be  in  a 
position  to  collect  and  report  it. 
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Results  of  the  analysis  revealed  several  high-value 
offices  that  cost  very  little  and  some  low-value  offices 
thaL  were  relatively  expensive.  The  analysis  also  drew 
attention  to  less  productive  offices  which  management  then 
examined  in  terms  of  other  possible  representational  and 
political  benefits  that  they  contributed  to  the  attache 
system.  (H#  2.2.1) 

5 . 5 Defense  Intelligence  Agency  Intelligence  Requirement^ 

Analysis 

The  collection  of  intelligence--always  important,  but 
alsfy  expensive--poses  the  following  difficult  question: 

What  is  the  value  of  acquiring  information  beyond  that 
already  available?  A corollary  question  is  whether  the 
value  of  acquiring  additional  information  justifies  the 
expenditure  of  funds  and  at  what  levels.  The  decision-maker 
often  does  not  realize  that,  in  making  a particular  decision, 
the  difference  between  available  and  even  perfect  information 
is,  in  fact,  a small  one.  Moreover,  his  uncertainty  about 
the  information  that  he  does  noi  have  and  about  its  value 
often  exceeds  the  likelihood  that  such  information  would 
alter  any  decision  he  might  make  on  the  basis  of  presently 
available  information.  An  on-going  project  has  develope 
decision  models  to  improve  the  identification  of  these  _ 
differences  and  the  evaluation  of  their  impact  on  the  decision- 
making process.  Emphasis  in  these  models  is  on  the  methods 
of  identifying  the  decision-maker's  needs  in  terms  of  the 
value  of  the  intelligence  to  him.  (IV,  4.3  and  also  References 
11.2.1.5  and  11.2.2.9  in  the  Bibliography) 
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6.0  UTILITY  ANALYSIS  IN  TREATY  NEGOTIATIONS 


Many  believe'  that  an  issue-by-issue  negotiation  strategy 
on  treaties  dealing  with  complex  international  problems  can 
result  in  nations  concluding  sub-optimal  agreements.  As  an 
alternative,  the  application  of  utility  analysis  to  negotia- 
tion problems  permits  simultaneous  consideration  of  ell 
issues  in  order  to  determine  which  of  the  possible  treaties 
would  res'ii  ^ in  each  party's  conceding  least  on  the  objectives 
it  values  the  most.  This  research  demonstrates  how  multi- 
attribute utilities  make  it  possible  to  consider  trade-offs 
among  the  issues  and  to  reduce  *he  sets  of  all  possible 
outcomes  ti'  the  Pareto-optimal  s>et.  (Ill,  2.2) 

6 . 1 Arms  Treaty  Negotiations 

A multi-attribute  utility  model  was  developed  to  handle 
th.e  attributes  of  strategic  weapon  systems  and  related 
issues  in  negotiations.  A major  problem  ^ or  anaiysis  was 
whether  or  not  the  operational  and  technical  characteris tics 
of  offensive  and  defensive  systems  equated  to  damage-causing 
and  damage- 1 imi ting  potential  were  adeqaute  as  determinants 
of  utility.  A preliminary  conclusion  is  that  more  knowledge 
is  needed' \’ith  respect  to  how  nations  perceive  the  strategic 
balance  and  to  its  effect  on  utility.  Two  workshops  were^  ^ 

conducted  which  addressed  perceptions  of  the  military  balance.  ^ 

(See  Section  10.0)  | 

;j 

6 . 2 Panama  Treaty  Negotiations 

Since  an  issue-by-issue  negotiating  strategy  often 
founders  (or  flounders)  on  legal  ar d ethical-historical 
matters,  intelligence  analysts  supporting  the  U.S.  teams 
negotiating  a new  canal  treaty  with  Panama  proposed  a better 
strategy.  Moreover,  on  the  assumption  that  a treaty  has  a , 

better  chance  of  winning  acceptance  and  continuing  compliance  i 

with  its  provisions  if  its  value  is  maximized  for  both 
parties,  a multi-at tribute  utility  model  was  constructed. 

This  modi  ^ relates  the  separate  utilities  of  each  issue, 
weights  th j importance  of  each  issue,  and  arrives  at  a ^ 

Paietian  optimal  boundary  which  defines  the  maximum  joint 
utilities  of  various  possible  treaties,  rince  the  aggregate 
joint  utilities  are  composed  of  the  comp  ementary  utilities 
for  both  parties,  a point  which  marks  ^'caal  and  unequal) 

utilities  for  both  parties  can  be  readi  .y  idenv-ified.  i 

By  distinguishing  legal  and  ethicc.l-historical  matters  | 

from  the  utilities  for  the  issues,  the  model  helps  negotiators  , 

avoid  snags  and  reiterations  in  the  negotiating  process. 

For  instance,  although  the  present  treaty  gives  the  U.S.  _ . 

canal  rights  in  perpetuity,  the  real  utility  of  the  duration  j 


I 


I 


1; 


However,  since  tne  uT*.Lity  ovei.  xcaaej. 

gradually,  it  is  po°iible  to  realize  some  benefits  in  terms 
of  the  duration  of  these  rights  for  increased  utilities  of 
oth^r  issues.  A milti-attribute  utility  model  permits  such 
trade-offs  so  as  to  assure  each  party  that  it  will  attain 
maximum  realizable  value.  This  study  has  served  as  a valuable 
tool  for  U.S.  negotiating  teams  in  formulating  U.S.  policies 
and  has  been  accepted  by  se.-ior  authorities.  New  studies 
Qj-0  being  conducted  to  develop  multi— attribute  utility 
models  for  multi-national  negotiations.  (II,  2.1.1,  and  also 
References  11.2.1.8  and  11.2.1.9  in  the  Bibliography) 
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7.0  BASIC  RESEARCH 
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Because  decision  analysis  depends  upon  theoretical 
research  undertaken  in  many  fields/  such  as  psychology/ 
mathematical  logic / systems  analysis/  and  organizational 
behavior/  further  research  must  be  conducted  to  make  the 
findings  in  those  fields  applicable  to  Department  of  Defense 
(DoD)  problems. 

7. 1  Multi- \ttribute  Utility  (MAU)  Analysis 

Decision  analysts  have  many  options  when  faced  with  the 
prol  lem  of  analvzing  decisions  whose  consequences  may  be 
eva:uated  by  multiple  criteria.  Research  has  helped  clarify 
these  options  and  provided  guidance  on  when  they  should  be 
exercised.  Research  began  with  a survey  of  the  literature 
in  the  field  and  was  continued  with  a systematic  review  of 
options  for  assigning  values  to  multi-attributed  objectives. 

Each  analytical  option  was  evaluated  in  the  context  of  a 
limited  number  of  problems  in  the  areas  of  defense/  foreign 
policy/  and  business.  (Reference  11.2.2.3  in  the  Bibliography) 

An  initial  finding  of  the  foiegoing  study  was  that 
reference  gambles  to  evaluate  utility  functions  are  not  sc 
promising  as  a direct  rating  procedure.  ; nother  finding 
indicated  that  the  complexity  of  the  ucility  scale  is  related 
to  the  length  of  time  over  which  events  are  modeled?  a 
short-time  forizon  typically  involves  complex  MAU  scales  and 
a long-time  horizon  simple  MAU  scales.  Conversely/  in  these 
instances/  the  probability  relationships  are  simple  and 
complex/  respectively.  Other  bindings  indicated  the t certain 
problems  with  lists  of  dimensions  combined  by  a formula 
(i.e./  weighted  sum)  may  be  reduced  by  holistic  assessment 
of  undecomposed  consequences  and  that  precise  definitions  of 
individual  scales  and  certain  evaluation  techniques  require 
further  study  (IV/  2.1) 

7 . 2  Modeling  Subsequent  Acts  for  Decision  Analysi-^ 

Although  standard  decision  theory  operates  on  the 
assumption  that  acts  subsequent  to  the  initial  choice  are 
predictable/  conditonal  on  explicitly  modeled  uncertainties/ 
this  assumption  often  distorts  the  evaluation  of  initial 
opti  5ns / typically  by  undervaluing  information-seeking 
strategies.  A study  attempted  to  avoid  these  difficulties 
by : 

• Treating  subsequent  acts  as  events  with  probabilities 
based  on  partial  information/  or 
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• Not  modeling  them  explicitly/  but  conu^  .loning 

terminal  events  or  value  probabilities  directly  on 
partial  information.  (IV,  2.2  and  also  Reference 
11.2.2.2  of  the  Bibliography) 

7 . 3 Cross-Cultural  Study  of  Uncertainty 

A study  to  achieve  better  understanding  of  cross- 
cultural  differences  in  the  perception  of  probability  aimed 
to  identify  those  inter-personal  differences  that  are  based 
on  culture  and  those  that  are  not,  and  to  explore  the  psycho- 
iSgIcirmechanisms  related  to  those  <5i«£«ences  The  study 
includes  a review  of  the  literature,  discussions  with  experts, 
obsLvatioual  and  clinical  studies,  development  of  measuring 
instruments  using  interactive  computer  graphics,  formal 
experiments,  reporting  and  feedback.  UV,  2.J) 

7 . 4 Unilateral  Disclosure 

Because  negotiating  parties  are  uncertain  about  what 
inforLtion  should  and  should  not  be  divulged,  the  problem 
is  to  develop  strategies  for  effectively  witholding  cpr  dis- 
^locinrinformation.  The  study  undertook  to  categorize 
virions  negotTStion  conflicts  in  terms  of  their  game-theoretic 
asoects  and,  within  each  category,  to  develop  appropriat^ 
ruleL  The  target  of  this  research  is  the  formulation  of 
strategy  for  international  negotiations  and  is  basea  on  a 
review^of  the  literature,  analysis  of  case  studies,  and 
theoLtical  formulation  and  analysis  of  theories  of  games 
and  decisions.  (IV,  2.4) 
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8.0  INTERACTIVE  COMPUTER  GRAPHICS 


Prospective  users  of  new  analytical  techniques  tend  to 
consider  their  introduction  into  operational  environments 
both  unduly  complex  and  time-consuming.  Even  when  adapted 
to  on-going  undertakings,  the  departure  of  analysts  experienced 
with  the  new  techniques  too  often  means  a loss  of  expertise, 
which  initially  requires  a repetition  of  the  training  process 
for  their  reolacements . To  facilitate  the  adaptation, _ 
training,  and  retention  of  expertise  without  loss  of  time, 
interactive  graphic  computers  can  be  programmed  to  promote 
efficient  operational  use.  Programs  for  structuring  problems 
and  making  alternative  computations  have  been  developed  and 
demonstrated . 


8.1  General  Tool  Development 


In 

programs 
analysis 
of : 


the  absence  of  any  studies  which  categorize  computer 
with  respect  to  computational  aspects  of  decision 
problems,  DDI  undertook  a study  with  the  objectives 


Pj^Q(jucing  a guide  for  decision  analysts  wishing  to 
use  available  computer  programs. 


Providing  an  overview  of  the  state-of-the-art 
computer  applications  of  decision  analysis  techniques 
(Reference  11.2.2.14  in  the  Bibliography) 


A technical  report  described  each  existing  program  and 
identified  those  representing  the  current  state  of  the  art. 
(Reference  11.2.2.1  in  the  Bibliography)  The  report  also 
identified  important  areas  of  future  technological  improve- 
ments . 


Two  programs  are  concerned  with  probability  assessments 
of  categorical  events,  or  events  on  a continuum.  One.  inter- 
active program  involves  a proper  scoring  rule  tool  for  cali- 
brating analysts  engaged  in  assessing  categorical  events. 

Another  program  implements  a method  of  directly  assessing 
the  probabilities  of  events  lying  along  a continuum.  Both 
involve  computer  feedback  for  faster  and  more  meaningful 
results  with  CTREE — a general-purpose  computer  programming 
language  for  handling  decision  trees  and  similar  structures. 
Moreover,  in  employing  CTREE  for  the  analysis  of  complex 
problems  concerning  the  values  to  be  placed  on  various  l^i^ds 
of  intelligence,  DDI  is  collaborating  with  Stanford  Research 
Institute  (SRI)  to  develop  a method  of  automatically  generating 
and  analyzing  decision  diagrams  by  specifying  an  influence 
diagram.  (IV,  5.1  and  IV,  5.3) 
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A third  proqram  on  the  direct  assessment  of  continuous 
probabilities  involved  the  use  of  an  interactive  graphic 
computer  program.  (Reference  11. 2. 2.1  in  the  Bibliography) 
Previous  researcli  indicated  that  individuals  more  readily 
judge  the  point  at  which  they  are  indifferent  between  two 
quantities  than  judge  the  relative  magnitude  of  the  two. 

Thus,  rather  than  assessing  the  probability  of  an  interval 
of  the  continuum  directly,  the  analyst  divides  the  continuu 
into  two,  three,  or  four  intervals  so  that  the  probabilities 
of  each  are  equal.  The  problem  with  sketching  a cumulative 
probability  distribution  on  graph  paper  is  that  the  time 
involved  delays  feedback  to  the  .nalyst.  However , an  interac- 
tive graphic  computer  program  can  provide  far  quicker  feedback 
By  estimating  tri-sections,  quadri-sections , and  credible 
intervals,  the  analyst  can  develop  a graph  representing  the 
direct  assessment  of  continuous  probabilities.  (TI,  2.3.1) 


8.2  Strategic  Systems  Analysis  Technology 

The  dec isi.on-maker  and  analyst  are  often  required  to 
evaluate  the  effects  of  alternative  assessments  which  may 
vary  over  a considerable  range.  The  defensive  and  offensive 
capabilities  of  strategic  weapon  systems  pose  a multi-  _ 
attribute  problem  in  assessment--the  analysis  of  which  is 
greatly  facilitated  by  using  an  interactive  graphic  computer. 

In  this  application,  the  analyst  operating  the  graphic 
display  can  vary  the  technical  parameters  of  the  strategic 
weapon  systems  belonging  to  one  or  both  nations  and  note  the 
overall  impact  upon  their  relative  strengths.  (Reference 
11.2.2.1  in  the  Bibliography)  This  technique  includes  the 
capability  to  vary  probabilities  for  offensive  and  defensive 
technical  improvements  and  thereby  provides  assessments 
useful  in  establishing  credibility  over  any  desired  distribu- 
tions. The  related  development  of  a "virtual  table  componen+- 
facilitates  programming,  updating  and  display;  it  does  so, 
in  part,  by  reducing  the  programming  effort  by  90%.  (IV, 

5.2) 
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9.0  HANDBOOK  FOR  DECISION  ANALYSIS 


Althouqh  it  is  now  winning  recognition  as  a useful  tool 
in  dealing  Sith  a wide-ranging  variety  ' 

dLision  Lalysis  was  for  K,any  years  a f ^ 
of  study.  As  a new  and  theoretical  field,  its  development 

was  piecemeal,  its  findings  were  scattered  .^ity 

number  of  highly  specialized  journaxs,  and  it^  applxcabiiity 
was  limited.  Out  of  this  disarray  has  emerged  a well 
integrated  normative  theory  about  the 

which  has  permitted  its  increas’i.g  Y^its 

kinds  of  problems.  The  coherence  of  the  theory  and  its 

accumulated  applications  to  f ■ ts  SsiJhts 

gence,  problems  have  enabled  the  codification  o its  insights 

Ld  techniques  into  a handbook  serving  the  needs  of  military 
decision-makers . 

A handbook  designed  for  the  decision-maker  and  his 
£?taff  rather  than  for  decision  analysts,  was  written, 
pubUsheS  and  widely  distributed  within  the  Defense  Intelli- 
gence  Community  and  other  Government  Agencies.  The  topics 
included  in  the  handbook  are,  among  others: 

models,  assessment  of  value,  utility  analysis  and  attitude 
toward  risk,  probability  assessments,  Bayes  Theorem,  and 
the  value  of  information.  The  handbooK  was  used  ^he 
Defense  Intelligence  School  in  the  training  of  intelli^^^ 
specialists.  It  has  also  been  used  the  United  States  ^ 
Force  Academy  in  a course  on  management.  Although  a roll 
UP  surve7o?  users  of  the  handbook  indicates  satisfaction 
with  it,  a revision  incorporating  recent  developments 
emphasizing  even  m,re  strongly  the  applicability  of  ^ecis 
analvsis  to  a wide  variety  of  problems,  rather  than  the 
decision  analytical  techniques  themselves,  is  J ’ 

(I,  1.2.5;  I,  4.0;  II,  4.0;  III,  4.0;  and  IV,  7 0 and  also 

Reference  11.2.2.8  in  the  Bibliography) 


10.0  WORKSHOPS,  BRIEFINGS,  SEMINARS,  AND  CONFERENCES 


Two  workshops  were  held  in  November  1973  to  introduce 
the  methodology  and  application  of  decision  theory  first  to 
intelligence  specialists  and  the  second  to  budget  and  program 
personnel.  Particular  emphasis  was  placed  upon  the  applica- 
tion of  decision  theory  models  to  analyses  of  the: 

• Effectiveness  of  strategic  forces, 

• Dependence  of  decision-making  on  intelligence 
information,  and 

• Relative  cost  of  intelligence  information  to  the 
intelligence  needs  of  high-level  decision-makers 
for  improving  strategic  force  effectiveness. 

In  January  1975,  DDI  conducted  the  second  of  two  related 
workshops  on  the  role  of  perceptions  of  the  military  balance 
in  the  DoD  decision-making  and  planning  processes.  Although 
there  was  some  divergence  of  opinion  with  respect  to  the 
degree  to  which  something  can/should  be  done  about  perceptions 
of  the  military  balance  in  the  decision-making  and  planning 
pj-Qcesses  of  DoD,  there  was  also  general  concurrence  that 
there  is  a need  for  additional  research  on  the  subject,  and 
a number  of  likely  areas  were  identified.  There  was  also 
general  concurrence  on  the  need  to  develop  an  overview  and 
general  plan  for  making  all  of  the  perceptions-related 
research  available  to  the  senior  decision-makers  and  planners 
in  a form(s)  that  they  can/will  use. 

Over  thirty  briefings,  seminars,  and  conferences  pertain- 
ing to  decision  analysis  were  conducted  or  attended  during 
the  period  October  1,  .972,  through  June  30,  1975.  Briefings 
on  the  methodology,  applications,  and  results  of  decision 
analytic  case  studies  were  tailored  to  suit  the  needs  and 
interests  of  various  individuals  and  organizations,  among 
which  were  the  NSC  staff,  the  Chairman  of  the  CIEP,  the 
Joint  Chiefs  of  Staff  (JCS) , and  the  DIA  staff.  Seminars 
and  conferences  included,  among  others,  the  CIA  Senior 
Seminar  and  the  Defense  Advanced  Research  Projects  Agency 
(ARPA)  simimer  study  oi.  Rese.  ’ch  Needs  for  Defense  Decision 
Processes.  The  central  focus  of  the  latter  seminar , ^ to 
which  DDI  contributed  on  matters  pertaining  to  decision 
theory,  was  upon  developing  through  research,  and  then 
implementing,  improved  decision-making  processes.  A DDI 
review  of  a Soviet  text  on  decision-making  procedures  reflects 
the  Soviet  belief  that  urgent  improvements  are  needed  in 
decision-making  tools  for  the  control  of  men  and  weapons. 

(I,  1.2.4;  I,  3.0;  II,  2.3.1;  III,  2.3;  III,  3.0;  III, 

Appendix  I;  and  IV,  6.3) 


f 


I 


19 


11.0  BIBLIOGRAPHY 


This  bibliooraphy  liscs  all  documents  referred  to  in 
the  text  of  this  report  and  categorizes  them  in  three  groups; 
namely,  1)  the  four  technical  reports  by  Decisions  and 
Designs,  Incorporated  (DDI)  on  progress  under  Office  of 
Naval  Research  (ONR)  Contract  N00014-73-C-0149 ; 2)  unsigned 
reports  listed  alphabetically  by  title;  and  3)  signed  reports 
listed  by  author. 

The  following  abbreviations  are  used  herein: 

CIA  - Central  Intelligence  Agency 

CIEP  - Council  on  International  Economic  Policy 

ARPA  - Defense  Advanced  Research  Projects  Agency 

DDI  - Decisions  and  Designs,  Incorporated 

DIA  - Defense  Intelligence  Agency 

ONR  - Office  V f Naval  Research 

RADC  - Rome  Air  Development  Center 

11.1  DDI  Technical  Reports  ^..ider  ARPA  Order  No.  2271  (7  Aug- 
gust  1972) . 

11.1.1  [I]  "First  O’o^rterly  Technical  Report  for  Quarter 
ending  31  January  1973." 

11.1.2  [II]  "Technical  Progress  Report  No.  2 for  1 February  - 
31  August  1973."  (Classified) 

11.1.3  [III]  "Technical  Progress  Report  No.  3 for  1 Septem- 
ber 1973  - 28  February  1974." 

11.1.4  [IV]  "Technical  Progress  Report  'Jo.  4 for  1 March 
1974  - 31  January  1975." 

11.2  Technical  Research,  Studies,  and  Reports 
11.2.1  Unsigned  material 

11.2.1.1  "An  Appraisal  of  the  Probability  of  an  Out- 
break of  Hostilities  on  Cyprus,  1 November  1972  - 30  April  1973," 
Special  Intelligence  Project,  for  the  Directorate  of  intelligence 
DIA.  (Classified) 

11.2.1.2  "An  Appraisal  of  the  Probability  of  a Soviet 
Attack  Across  the  Sino-Soviet  Border,  1 November  1972  - 31  October 
1973,"  Special  Intelligence  Project,  for  the  Directorate  of  Intelli 
gence,  DIA.  (Classified) 

11.2.1.3  "A  Bayesian  Approach  to  Future  Systems  Assess- 
ment," CIA  Scientific  and  Technical  Intelligence  Report  (February 
1974).  (Classified) 


20 

I 

k 


esjc 


11  2 1.4  -computer  Sale  to  the  Soviet  Blrc,"  DDI 
Technical  RepiiJ'73-4  (October.  1973),  to  the  CIEP.  (Classified) 

for^th^^eS.ii  Studies 'Branch,  DIA,  and  Ar'A.  (Classified) 

11  2 1 6 "Decision  Theoretic  Procedures  for  Intelli- 
gence,”  Section  iv,  RADC-TR-73-406  (February,  1974),  for 
RADC.  (Classified) 

sfMiuC?rof?SnJi«\fsou?h'’5^^^^ 
f«L-^ri9f4^2rs:rtereri97i^^“^pecl^^^  -03ect, 

for  the  Directorate  of  Intelligence,  DIA.  (Classified) 

11  2.1.8  "Panama:  1973  - The  Year  of  the  ireaty. 

(28  November  1972),  for  a CIA  Directorate  of  Intelligence  Memorandum. 

(Classified) 

11  2 1 9 "The  Panama  Canal  Negotiations  - A Methodologi- 
cal Approach"  (21  August  1974),  a CIA  Intelligence  Report.  (Classi- 
f ied) 

11.2.2  Signed  material 

n 9 2 1 Barclay,  S.,  and  Randall,  S.,  "Graphics  In- 
teractive  Sy^i;miii  o^Sfsioi;' The  retie  Analysis."  DDI  Technical 
Report  (30  December  1974),  for  RADC. 

n 9 2 2 Brown,  R.  V.,  "Modeling  Subsequent  Acts  for 
Decision  Analysis,"  DDI  Technical  Report  No.  75-1  (May 

,1223  Brown,  R.  V.  and  Hatch,  J.  E.,  "Toward  a 

Bayesian  Mode^'  Hierl“^h(cal  Infe«n;e?"An^S||i4^ 

Behavior  and  Human  Per f orm_ance , Vol.^10,  No.  J, 

London:  Academic  Press,  December,  1973). 

11  2 2.5  Kelly,  C.  W.  Ill,  and  Barclay,  S.,  "Im- 

?e~"  frocSngfoftiH97fReuSn^  aX MaLSnability 

Symposium,  IEEE. 

11  2 2 6 Kelly,  c.  W. , HI,  Brown  R.  V.,  Stewart, 

11. 2.  ^.b  y,  of  NATO  Response  to 

R.  R. , and  Blvi  a,  Attack  " DDI  Technical  Report  74-1 

and  Impending  Warsaw  Pact  Attack, 

(August  1974) . 


21 


I 


11.2.2.7  Peterson,  C.  R . , Brown,  R . V.,  and 
Gustafson,  D. , "Allocation  of  Decision  Analysis  Resources," 

DDI  Technical  Report  75-3  (c.  January,  197G)  for  ARPA. 

11.2.2.8  Peterson  C.  R.  , I'elly,  C.  V. . , lii/  Barclay, 
S.,  Brown,  R.  V.,  and  Hazard,  T.  H.,  Handbook  for  Decision  Analysj^ 
(16  chapters  dated  individually  from  June  through  October,  1973). 

11.2.2.9  Peterson,  C.  R. , e t . al . , "Inference  from 

Evidence:  Bayes'  Theorem, " Chapter  12,  Handbook  for  Decision 

Analysis , (see  11.2.2.8  ahcve) . 

11.2.2.10  Peterson,  C.  R. , et . al . , "Hierarchical 
Inference,"  Chapiter  14,  Handbook  for  Decision  Analysis,  (see 
11.2.2.8  above) . 

11.2.2.11  Peterson,  C.  R.  , Kelly,  C.  V.  a]^, 

"Multiple-Stage  Probabilistic  Inforniation  Processing,  in 
Organizational  Behavior  and  Hur.an  Performance,  Vol^_^10,  No.  3, 

■(New  York  and  London:  /cademic  Press,  December,  1973^. 

11.2.2.12  Peterson,  C.  R.  and  Hazard  T.  H.,  "Odds 
Versus  Probabilities  for  Categorical  Lvents,"  DDI  Technical  Re- 
port 73-2  (July,  1973). 

11.2.2.13  Peterson,  C.  R.  and  Barclay,  E.,  "Two 
Methods  for  Assessing  Probability  Distribution,"  DDI  Technical 
Report  73-1  (August,  1973). 

11.2.2.14  Ulvila,  J.  V’.,  "A  Pilot  Survey  of  Com- 
puter Programs  for  Decision  Analysis,"  DDI  Technical  Report 
75-2  (January,  1975). 


DISTRIBUTION  LIST 


Department  of  Defense 


Director  of  Net  Asiu-^ssr  ent 
Office  of  the  Secretary  of  Defense 
The  Pentagon,  Roc m 3A930 
Washington,  DC  20301 

Assistant  for  Resource  Evaluation 
Office  of  the  Deputy  Assistant  Sec. 
of  Defense  (Program  Analysis  and 
Evaluation) 

The  Pentagon,  Room  2E326 
Washington,  DC  20301 

Special  Assistant  to  the  Principal 
Deputy  Assistant  Sec.  of  Defense 
(International  Security  Affairs) 

The  Pentagon,  Room  AE825 
Washington,  DC  20301 

Assistant  Director  (Net  Technical 
Assessment) 

Office  of  the  Deputy  Director  of 
Defense  Research  and  Engineering 
(Test  and  Evaluation) 

The  Pentagon,  Room  3C123 
Washington,  DC  20301 

Assistant  Director  (Environment  and 
Life  Sciences) 

Office  of  the  Deputy  Director  of 
Defense  Research  and  Engineering 
(Research  and  Advanced  Technology) 
Attention:  Lt.  Col.  Henry  L.  Taylor 

The  Pentagon,  Room  3D129 
Washington,  DC  20301 

Office  of  the  Assistant  Sec.  of  Defense 
(Intelligence) 

Attention:  CDR  Richard  Schlaff 

The  Pentagon,  Room  3E279 
Washington,  DC  20301 


Office  of  the  Assistant  Secretary 
of  Defense  (Intelligence) 
Attention:  Mr.  Morton  E.  Goulder 

The  Pentagon,  Room  3E282 
Washington,  D.C.  20301 

Director,  Weapon  Systems  Evaluation 
Group 

400  Army  Navy  Drive 
Arlington,  VA  22202 


Chief  of  the  Studies,  Analysis, 
and  Gaming  Agency 

J-5,  Office  of  the  Joint  Chiefs  of  Staff 
The  Pentagon,  Room  1D940 
Washington,  DC  20301 

Director,  Defense  Advanced  Research 
Projects  Agency 
1400  Wilson  Boulevard 
Arlington,  VA  22209 

Director,  Human  Resources  Research 
Office 

Defense  Advanced  Research  Projects  Agency 
1400  Wilson  Boulevard 
Arlington,  VA  22209 

Director,  Technology  Assessments  Office 
Defense  Advanced  Research  Projects 
Agency 

1400  Wilson  Boulevard 
Arlington,  VA  22209 

Director,  Program  Management  Office 
Defense  Advanced  Research  Projects 
Agency  (ARPA) 

1400  Wilson  Boulevard 
Arlington,  VA  22209 
(two  copies) 

Director,  ARPA  Regional  Office 
(Europe) 

Headquarters,  U.S.  European  Command 
APO  New  York  09128 

Director,  ARPA  Regionax  Office 
(Pacif ic) 

Staff  CINCPAC,  Box  13 
FPO  San  Francisco  96610 


Director,  Program  Management  Office 
Defense  Systems  Management  School 
Building  202 
Ft.  Belvoir,  VA  22060 

Chairman,  Dept,  of  Curriculum 
Development 
National  War  College 
Ft.  McNair,  4th  and  P Street,  S.W. 
Washington,  DC  20319 


Documentation 


Department  of  National  Security 
Management  Studies 
Attention:  Mr.  Victor  J.  Baran 

Industrial  College  of  the  Armed  Forces 
Ft.  McNair,  4th  and  V Street,  S.W. 
Washington,  DC  20319 


Chief  of  the  Plans,  Research,  and 
Evaluation  Branch  (IS-IC) 
Defense  Intelligence  School 
Washington,  DC  20374 


Chief  of  Staff 
Defense  Intelligence  Agency 
Attention:  RADM  Donald  P. 

The  Pentagon,  Room  3E261 
Washington,  D.C.  20301 


Harvey 


Administrator,  Defense 
Center 

Attention:  DDC-TC 

Cameron  Station 
Alexandria,  VA  22314 
(12  copies) 


Deputy  Director,  National  Security  Agency 
Attention:  Mr.  Benson  K.  Buffham 

Ft.  George  Meade,  MD  20755 


Department  of  the  Army 


I 

I 

i 

( 


)uty  Under  Secretary  of 
(Operations  Research) 
s Pentagon,  Room  2E621 
eshincton,  DC  7 ''310 


the  Army 


Director,  Systemi  Review  and  Analysis 

Office  ,, 

Office  of  the  Deputy  Chief  of  Staff 
for  Research,  Development  and 
Acquisition 
Department  of  the  Army 
The  Pentagon,  Room  3E426 
Washington,  DC  20310 


Commander,  U.S.  Array  Intelligence 
Threats  Analysis  Detachment 
Arlington  Hall  Station 
4000  Arlington  Boulevard 
Arlington,  VA  22212 


Director,  Army  Library 
Army  Studies  (ASDIRS) 

The  Pentagon,  Room  1A534 
Washington,  DC  20310 


Director,  U.S.  Army  Institute  for 
the  Behavioral  and  Social  Sciences 


1300  Wilson  Boulevard 
Arlington,  VA  22209 


Associate  Director,  U.S.  Army  Institute 
for  the  Behavioral  and  Social  Sciences 


1300  Wilson  Boulevard 
Arlington,  VA  22209 


Director,  Organization  and  Systems 
Research  Laboratory 
U.S.  Army  Institute  for  the  Behavioral 
and  Social  Sciences 
1300  Wilson  Boulevard 
Arlington,  VA  22209 

Director,  Methodology  and  Resources 
Directorate 

U.S.  Army  Concepts  Analysis  Agency 
8120  Woodmont  Avenue 
Bethesda,  MD  20014 

Director,  Strategic  Studies  Institute 
U.S.  Army  Combat  Developments 
Command 

Carlisle  Barracks,  PA  17013 

Director  of  Educational  Technology 
(AMXMC-ET) 

Army  Logistics  Management  Center 
Ft.  Lee,  VA  23801 

Assistant  to  the  Dean  for  Plans 
and  Programs 

United  States  Military  Academy 
West  Point,  NY  10996 

Chief,  Studies  and  Analysis  Office 
Headquarters,  Army  Training  and 
Doctrine  Command 
Ft.  Monroe,  VA  23351 

Commander,  U.S.  Army  Research  Office 
(Durham) 

Box  CM,  Duke  Station 
Durham,  NC  27706 


.■1 

'f 

I 

-I 


fe  . . 


♦ 


24 


Commanding  Officer 
U.S.  Army  Foreign  Science  and 
Technology  Center 
700  East  Jefferson  Street 
Charlottesville,  VA  22901 


Education''!  \dvisor 
Array  Intelligence  School 
Ft.  Huachuca,  Arizona  85613 


Department  of  the  Nav\ 


Office  of  the  Chief  of  Naval 
Operations  (OP-987P10) 
Attention:  Dr.  Robert  G.  Smith 

and  Mr.  Paul  Conway 
Washington,  DC  20350 
(two  copies) 

Office  of  the  Chief  of  Naval 
Operations  (OP-951) 

Washington,  DC  20350 

Office  of  the  Chief  of  Naval 
Operations  (OP-987) 

Washington,  DC  20350 

Director,  Engineering  Psychology 
Programs  (Code  455) 

Office  of  Naval  Research 
800  North  Quincy  Street 
Arlington,  VA  22217 
(three  copies) 

Office  of  Naval  Research 
(Code  230) 

800  North  Quincy  Street 
Arlington,  VA  22217 

Office  of  Naval  Research 
Naval  Analysis  Programs 
(Code  431j 

800  North  Quincy  Street 
Arlington,  VA  22217 

Office  of  Naval  Research 
Operations  Research  Programs 
(Code  434) 

800  North  Quincy  Street 
Arlington,  VA  22217 

Office  of  Naval  Research 
(Code  436) 

Attention:  Dr.  Bruce  MacDonald 

800  North  Quincy  Street 
Arlington,  VA  22217 


Office  of  Naval  Research 
Information  Systems  Program  (Code  437) 
800  North  Quincy  Street 
Arlington,  VA  22217 

Office  of  Naval  Research  (ONR) 
International  Programs  (Code  1021P) 

800  North  Quincy  Street 
Arlington,  VA  22217 

Director,  ONR  Branch  Office 
Attention:  Psychologist 

536  South  Clark  Street 
Chicago,  IL  60605 

Director,  ONR  Branch  Office 
Attention:  Dr.  J.  Lester 

495  Summer  Street 
Boston,  MA  02210 

Director,  ONR  Branch  Office 
Attention:  Dr.  E.  Gloye  and 

Mr.  R.  Lawson 
1030  East  Green  Street 
Pasadena,  CA  91106 
(two  copies) 

Director,  Naval  Research  Laboratory 
Technical  Information  Division 
(Code  2627) 

Washington,  DC  20375 
(six  copies) 

Director,  Naval  Research  Laboratory 
(Code  2C29) 

Washington,  DC  20375 
(six  copies) 

Scientific  Advisor 
Office  of  the  Deputy  Chief  of  Staff 
for  Research,  Development  and  Studies 
Headquarters,  U.S.  Marine  Corps 
Arlington  Annex 
Columbia  Pike 
Arlington,  VA  20380 


25 


1 


Headquarters,  Naval  Material  Command 
(Code  0331) 

I Attention:  Dr.  Heber  G.  Moore 

J Washington,  DC  20360 

'I  Headquarters,  Naval  Material  Command 

;j  (NAVMAT  03AA) 

Ji-  Attention:  Mr.  Arnold  Rubinstein 

I Washington,  DC  20360 

Commander,  Naval  Electronic  Systems 
i Command 

Command  and  Control  Division 
! (Code  530) 

i|  Washington,  DC  20360 

Commander,  Naval  Electronic  Systems 
^ ^ Command 

■I  Support  Systems  Office  (Code  9A71) 

Washington,  DC  20360 

i 

, , Human  Fac  tors  Engineering  Branch 

Crew  Systems  Department 
' Naval  Air  Development  Center 

! . Attention:  CDR  Pobert  Wherry 

:]|  Johnsville 

i i Warminster,  PA  1897A 

? 1 

I i- 

.1  Commander,  Naval  Facilities  Engineering 

‘ i Command 

; Plans  and  Programs  Division  (Code  031A) 

'j  Alexandria,  VA  22332 

: * Naval  Medical  Research  and  Development 

Command  (Code  AA) 

j Attention:  CDR  Paul  Nelson 

Naval  Medical  Center 
! Bethesda,  MD  2001A 

; I Head,  Human  Factors  Division 

i:  ; Attention:  Mr.  Richard  Coburn 

j;  i Naval  Electronics  Laboratory  Center 

; San  Diego,  CA  92152 

; ( 

l;  Human  Factors  Engineering  Branch 

^ : (Code  53A2) 

: Attention:  LCDR  Robert  Kennedy 

j U.S.  Naval  Missile  Center 

' , Point  Mugu,  CA  930A2 


Dean  of  Research  Administration 
Naval  Postgraduate  School 
Monterey,  CA  939A0 

Navy  Personnel  Research  and  Development 
Center,  Management  Support  Dept. 

(Code  210) 

San  Diego,  CA  92152 

Naval  Personnel  ’Research  and 
Development  Center  (Code  305) 

Attention:  Dr.  Claries  Gettys 

San  Diego,  CA  92152 

Human  Factors  Department  (Code  N215) 

Naval  Training  Equipment  Center 
Orlando,  FL  32813 

Office  of  the  Oceanographer  of 

the  Navy  j 

Attention:  Dr.  Andreas  B.  Rechnitzer  j 

Hoffman  Building  II  I 

200  Stovall  Street  ! 

Alexandria,  VA  22332  i 

Training  Analysis  and  Evaluation  ^ 

Group  I 

Naval  Training  Equipment  Center 
(Code  N-OOT) 

Attention:  Dr.  Alfred  F.  Smode 

Orlando,  FL  32813 

Technical  Director 
Naval  Intelligence  Support  Center 
A301  Suitland  Road 
Suitland,  MD  20390 

Director,  Center  for  Advanced  Research, 

Naval  War  College 
Attention:  Prof.  C.  Lewis 

Newport,  RI  028A0 

Dean  of  the  Academic  Departments 
U.S.  Naval  Academy 
Annapolis,  MD  21A02 

Chief,  Intelligence  Division 
Marine  Corps  Development  Center 
Quantico,  VA  23701 


J 


Department  of  the  Air  Force 


Deputy  Assistant  Chief  of  Stafl  for 
Studies  and  Analysis 
Department  of  the  Air  Force 
The  Pentagon,  Room  1E388 
Washington,  DC  20330 

Assistant  for  Requirejnents  Development 
and  Acquisition  Programs 
Office  of  the  Deputy  Chief  of  Staff  for 
Research  and  Development 
Department  of  the  Air  Force 
The  Pentagon,  Room  4C331 
Washir.'iton,  DC  20330 

Air  Force  Office  of  Scientific  Research 
Life  Sciences  Directorate 
lAOO  Wilson  Boulevard 
Arlington,  VA  22209 

Director,  Air  University 
Maxwell  AFB,  AL  36112 

Dean  of  the  Academic  Departments 
U.S.  Air  Force  Academy 
Colorado  Springs,  CO  808A0 

Director  of  Intelligence  Applications 
Headquarters,  Air  Force  Systems  Command 
Andrews  AFB 
Washington,  DC  20331 


Chief,  Systems  Effectiveness  Branch 
Human  Engineering  Division 
Attention:  Dr.  Donald  A.  Topmiller 

Wright-Patterson  AFB,  OH  A5A33 

Director,  Foreign  Technology  Division 
Wright-Patterson  AFB,  OH  A5A33 

Chief  Scientist 

Air  Force  Human  Resources  Laboratory 
Attention:  Dr.  Howard  L.  Parris 

Brooks  AFB,  TX  78235 

Aerospace  Medical  Division 
(Code  RBH) 

Attention:  Lt.  Col.  John  Courtrlght 

Brooks  AFB,  TX  78235 

Commander,  Rome  Air  Development  Center 
Attention:  Mr.  Joh.i  Atk  nson 

Griffiss  AFB 
Rome,  NY  13^40 

IRD,  Rome  Air  Development  Center 
Attention:  Mr.  Frederic  A.  Dion 

Griffiss  AFB 
Rome,  NY  13440 

Director,  Advanced  Systems  Division (AFHRL/AS) 
Attention:  Dr.  Gordon  Eckstrand 

Wright-Patterson  AFB,  OH  45433 


Other  Government  Agencies 


Chief,  Strategic  Evaluation  Center 
Central  Intelligence  Agency 
Headquarters,  Room  2G24 
Washington,  DC  20505 


Director,  Center  for  the  Study  of 
Intelligence 

Central  Intelligence  Agency 
Attention:  Mr.  Dean  Moor 

Washington,  D.C.  20505 


Office  of  Life  Sciences 
Headquarters,  National  Aeronautics  and 
Space  Administration 
Attention:  Dr.  Stanley  Deutsch 

600  Independence  Avenue 
Washington,  DC  20546 


27 


Other  Institutions 


The  Johns  Hopkins  University 
Department  of  Psychology 
Attention:  Dr.  Alphonse  Chapanis 

Charles  and  34th  Streets 
Baltimore,  MD  21218 

Institute  for  Defense  Analyses 
Attention:  Dr.  Jesse  Orlansky 

400  Army  Navy  Drive 
Arlington,  VA  22202 

CTEC,  Incorporated 
Attention:  Mr.  Harold  Crane 

7777  Leesburg  Pike 
Falls  Church,  VA  22043 

Director,  Social  Science  Research 
Institute 

University  of  Southern  California 
Attention:  Dr.  Ward  Edwards 

Los  Angeles,  CA  90007 

Perceptronics,  Incorporated 
Attention:  Dr.  Amus  Freedy 

6271  Variel  Avenue 
Woodland  Hills,  CA  91364 

Director,  Human  Factors  Wing 
Defense  and  Civil  Institute  of 
Environmental  Medicine 
Attention:  Dr.  C.  H.  Baker 

P.O.  Box  2000 
Downsville,  Toronto 
Ontario,  Canada 

Columbia  University 
Department  of  Psychology 
Attention:  Dr.  Eugene  Gala,  ter 

New  York,  NY  10027 

Stanford  University 
Attention:  Dr.  R.  A.  Howard 

Stanford,  CA  94305 

Integrated  Sciences  Corporation 
Attention:  Mr.  Gary  W.  Irving 

1532  Third  Street 
Santa  Monica,,  CA  90401 


Mr.  Luigi  Petrullo 

2431  North  Edgewood  Street 

Arlington,  VA  22207 

Montgomery  College 
Department  of  Psychology 
Attention:  Dr.  Victor  Fields 

Rockville,  MD  20850 

General  Research  Corporation 
Attention:  Mr.  George  Pugh 

7655  Old  Springhouse  Road 
McLean,  VA  22101 

Grumman  Aerospace  Corporation 
Attention:  Mr.  J.  W.  Stump 

Bethpage,  NY  11714 

Oceanautics,  Incorporated 
Attention:  Dr.  W.  S.  Vaughan 

3308  Dodge  Park  Road 
handover,  MD  20785 

Journal  Supplement  Abstract  Service 
American  Psychological  Association 
1200  17th  Street,  N.W. 

Washington,  DC  20036 
(three  copies) 

Director,  Applied  Psychology  Unit 
Medical  Research  Council 
Attention:  Dr.  A.  D.  Baddeley 

15  Chaucer  Road 
Cambridge,  CB2  2EF 
England 

Department  of  Psychology 
Catholic  University 
Attention:  Dr.  Bruce  M.  Ross 

Washington,  DC  20017 

Stanford  Research  Institute 
Decision  Analysis  Group 
Attention:  Dr.  James  Matheson 

Menlo  Park,  CA  94025 

Hebrew  University 
Department  of  Psychology 
Attention:  Dr.  Amos  Tversky 

Jerusalem,  Israel 


28 


'■  imiiii»«bii  nxwa— If' I H[l  I' I > H 1 1 II  »|J  I 


Human  Factors  Research,  Incorporated 
Santa  Barbara  Research  Park 
Attention:  Dr.  Robert  R.  Machie 

6780  Cortona  Drive 
Goleta,  CA  93017 

University  of  Pennsylvania 
Wharton  School 
Attention:  Dr.  H.  L.  Morgan 

Philadelphia,  PA  19174 

Oregon  Research  Institute 
Post  Office  Box  3196 
Attention:  Dr.  Paul  Slovic 

Eugene,  Oregon  97403 


Eclectech  Associates,  Incorporated 
Post  Office  Box  1/9 
Attention:  Mr.  Ala."'  J.  Pesch 

North  Stonington,  CT  Oo359 

University  of  Michigan 
Department  of  Electrical  and  Computer 
Engineering 

Attention:  Professor  Kan  Chen 

Ann  Arbor,  Michigan  94135 


29 


UNCLASSIFIED 


■■.Fril  .iTY  CLASSIFICATION  OF  THIS  PAGE  (■>yh»n  D.l.  Enffd) 


REPORT  DOCUMENTATION  PAGE 


KfcAD  INSTKUCIIONS 
BKFORE  COWtM.ETING  KORM 


h bepo«t  (.uwacn 


I Bill  •lJUiriTl'* 

^ jOerl  Ion  Theory  Research « \ 


Tj  OOVl  iccFsSION  CATALOG  NUMBtB 

' 

/ Technical  Hep«t. 

I / Ortiai*»  1072-^uymt  ^7 

lAKOn*  niNtai  ii 

I \ \fl  DT/TR-75-5/  1(^0^, 

- 1 c n-tf  T »n  iTnrtt*  *tv*i°* 

'f  Knhni _7 1;.  '•  ' 


Cameron  R . /Veterson]/"^'  Nnoni4-75-r-f^42f*j  r 


‘V  Nfin014-73-C-014« 


) 


1 9 performing  organization  name  and  address 

Decisions  and  Designs,  Incorporated 
Suite  100,  7900  Westpark  Drive 
McLean,  Virginia  22101 


10  program  element,  PROJECT,  T ASK 
AREA  » WORK  UNIT  NUMBERS 


ARPA  Order  2271t> 
NR-197-J023 


|n,  controlling  office  name  and  address  / ,r,T,A  \i/T7T 

Defense  Advanced  Research  Projects  Agency (ARPA^Ji 

1400  Wilson  Boulevard 
Arlington,  Virginia  22209 


"u  monitoring  agency  name  4 ADDRESSfl/  dUUfnt  Itom  Conltolllnt  Olllct) 

Office  of  Naval  Research  (Code  455) 

800  North  Quincy  Street 
Arlington,  Virginia  22217 


^-^BCUBITAAi 
/A,  \ r,iO  - 


CLASS.  (ol  IM»  rtporl) 


SSif led 


iSa  declassification  downgrading 

SCHEDULE 


M6  distribution  statement  foMMa  RapofO 

Approved  for  public  release,  distribution  unlimited 


17.  distribution 


statement  (ollh,  a6a(rac(  anfarad  In  Block  30,  II  d///aran(  horn  Report) 


18  SUPPL EMEnT ARY  NOTES 


I 19.  KEY  WORTS  (Continue  on  ra.ar.a  eld.  II  neceery  -rd 

Decision  analysis  Evaluation 


Decision  theory 

Decision  analysis  research 

Assessment 


Forecasting 
Policy  formulation 
Value  of  information 


Negotiations 
Interactive  computer 
graphics 


ot  decision  analytic 

techniaues  by  Decisions  and  Designs,  Incorporated  (DDI)  during  the  period 
October  1,  1975  through  June  30,  1975.  As  such,  this  report  provides^a 
summary  of  those  activities  in  terns  of  the  following  categories: 

• Assessment  and  forecasting ^ 

• Evaluation  of  alternative  solutions  , 

• Policy  formulation  and  implementation  ^ 

• Value-of-information  analysis ^ 


DD  . ^°n"“  3 1473 


EDITION  OF  1 NOV  65  IS  OBSOLETE 

390 


t^<Z  ( 


UNCLASSIFIED 


SECURITY  CLASSIFICATION  OF  THIS  PAGE  fW.en  Da(a  Entered), J. 


mmrnm. 


UNCLASS I FIFO 


security  classification  of  this  PAGE(iyh»n  D.(.  F.nt,r,d) 

• L'tllity  analysis  in  treaty  nepoti  itiona^ 

• Basic  decision  analysis  research 

• Interactive  computer  graphics 

• A handbooV  for  decision  analysis  ; ' 

• Decision  analysis  workshops,  brie  ings,  seminars  and  conferences  ^ 

These  summaries  are  keyed  to  four  major  Technical  Progress  Reports 
by  DDT  and  to  other  relevant  references.  . 

\ 


UNCLASSIFIED 

security  classification  of  this  PAGE^lFhon  Dbib  Enlrred) 


f 


